Extracting regression rules from neural networks

This paper proposes a new framework and method for extracting regression rules from neural networks trained with multivariate data containing both nominal and numeric variables. Each regression rule is expressed as a pair of a logical formula on the conditional part over nominal variables and a polynomial equation on the action part over numeric variables. The proposed extraction method first generates one such regression rule for each training sample, then utilizes the kappa-means algorithm to generate a much smaller set of rules having more general conditions, where the number of distinct polynomial equations is determined through cross-validation. Finally, this method invokes decision-tree induction to form logical formulae of nominal conditions as conditional parts of final regression rules. Experiments using four data sets show that our method works well in extracting quite accurate and interesting regression rules.